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This article elaborates the result of the Pilot Study which is related to IT 
implementation factors at the Higher Education Institution (HEI), a pilot 
study is used to validate quantitative readiness model of IT implementation. 
The main objective of this study is examining the factors that influence 
the readiness of IT implementation in HEI. This study attempts to analyze IT 
Content factors, Institutional Context, People, Process, Technology, Service 
Quality and IT Implementation Readiness (ITIR). The sample of data was 
taken from 150 HEIs throughout Indonesia which was then processed in 
statistical techniques through PLS-SEM method. The research finding shows 
that 9 of the 14 hypotheses used as ITIR model construct have a very 
significant influence on IT implementation on HEI, so that this finding can 
provide a comprehensive contribution to the literature of ITIR model 


development. 
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1. INTRODUCTION 

Along with the rapid development of information technology nowadays, it is certain that IT 
is involved in all fields, including higher education institution [1]. IT does not only benefit the things that 
directly utilize the potential of technology, but also encourage the emergence of new innovations in doing 
work/activities [2]. One task that belongs to high education is the utilization of IT development in supporting 
the provision of high quality services of high education which is affordable for the people who need 
education [3]. There are two important ways which can lead us to the purpose of IT development. First, high 
education needs to understand the roles that IT can do in supporting higher education processes and the ways 
that IT can perform that roles. Second, understanding the way to build a conducive environment so that IT 
can provide optimal support [4]—[6]. 

John Ward and Joe Peppard stated that there are three main objectives of the application of SIAT 
in an organization. First of all is improving work efficiency by automating various processes that manage 
information. Secondly, improving management effectiveness by fulfilling information needs for decision 
making. Thirdly, improving competitiveness or improving an organization’s competitive advantage by 
changing the style and way of doing business. The fact that this application is not in line with expectations, 
the value of failure reaches 18%, the implementation of IT problems is 55% and successful IT 
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implementation is 27% [7]. The result of the study of the utilization of readiness for IT implementation have 
a significant impact on the success of IT project on HEI [8], [9]. 

This article constitutes a series of doctoral research that covers literature review, pretest and pilot 
study. The objective of pilot study is to validate the model of IT Implementation Readiness using quantitative 
method. Moreover, this study is intended to reveal the status of IT implementation readiness in HEI and to 
ascertain the factors that affect IT implementation readiness. In order to ascertain the above objectives, there 
was applied a statistical method called partial structural-structural equation modeling (PLS-SEM) with 
SmartPLS 2.0, which is considered to be appropriate for this study. The total responses of respondents 
(n = 180) were taken from some HEI respondents in Indonesia. 14 hypotheses as shown in Figure 1 were 
then tested. The results show that 8 hypotheses were rejected after the structural model assessment. 
The purpose of this study is exploring the factors that influence ITIR organizations. 

This study covers five sections. The first part of this article is introduction, followed by the literature 
review which reveal the modeling and pilot study. Meanwhile the third part is the research method applied in 
the pilot study, which is followed by the research findings and the data analysis. The following part is 
discussion and data presentation, that ends with the research conclusion 





Figure 1. The research model 


2. RESEARCH METHOD 
2.1. Research model 

As stated by Matti Synco, the factors that have an influence on the implementation of ICT in Higher 
Education (HEI) are divided into four groups, namely global challenge, benefits of technology; pedagogical 
factors; barrier and limitation to ICT application; and political issues. Moreover, Alemayehu Molla 
and Paul S. Licker highlighted the role and perception of organization based on the perception of e-readiness 
organization (POER) and perception of e-readiness environment (PEER), which included innovative, 
managerial, organizational, and environmental characteristics as determinant factors of IT adoption 
and implementation in HEI [10]. In this regard, the framework of e-readiness model which (1) can be reused 
consists of components that are not bound to a particular moment in time, and (2) includes all relevant 
variables [11], the variables include people, process and technology [12]. This indicator is easy to measure 
and has relevant factors for IT implementation in HEI [13], while e-readiness has become a core feature 
of international socio-economic development for its ability to change society movement from traditional 
relation to a more modern way of thinking or dealing with health, education and production [14], so HEI is 
able to have a smart campus with IT utilization indicators on HEI that are used in optimal and massive 
manners [15]. Another factor that influences the success of IT implementation in HEI is the quality of IT 
services adjusted to Institutional Context [16]. 

The modelling in this study is built by combining, adapting and adopting existing models using 
Input-Process-Output logic to produce a new model as shown in Figures 1 and 2. Through the combination 
and adoption, 42 indicators were used to measure IT implementation readiness in HEI as shown in Table 1. 
Previously there were 46 indicators interconnected, but their number was simplified into 42 indicators 
according to the recommendations of the conducted studies that aim to perform indicator validation, pre-test 
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and confirmation using quantitative method. In this study, an inductive-quantitative approach is conducted to 
validate the proposed model through survey results. 

Furthermore, this quantitative research is intended to validate the model, both logically 
and empirically. Logical validation is done when the model is logically analyzed in accordance with the 
content and aspects expressed, while empirical analysis works when an instrument can reveal all data 
captured by the five senses existing in the objects of the research field [17, 18]. In addition, this quantitative 
research is concerned with the degree of objectivity, consistency and stability of data or findings that are 
related to the developed model, regarding the degree of many people agreement towards data. Based on the 
obtained data, the point from this validation study is that the stakeholders participating in this study revised 
the first model by repositioning the variables so that the number of relationships between variables within the 
model increased. Meanwhile, within the other improvement, the researcher also reformulated the conceptual 
framework by accommodating the TESCA model as an additional dimension from the framework. 
In graphical illustration, Figure 3 shows the model a conceptual framework. 


Table 1. Reference of Indicators 




















Code Indicators References 
ITC1 Timeliness (19],[20],[21],[11] 
ITC2 Completeness 

ITC3 Consistency 

ITC4 Relevance 

ITCS Technology Complexity 

ITC6 Information Quality 

ITC7 System Quality 

ITC8 Perceived Usefulness 

ITC9 Perceived Ease of Use 

INC1 Institutional Policies [19],[22],[10],[23] 
INC2 Management Involvement 

INC3 Infrastructure Availability 

INC4 External Environments 

INCS Legal Environment 

PPL1 Workforce Capability [12],[24],[25] 
PPL2 Leadership 

PPL3 Competency 

PPL4 Resources 

PPLS Change Management 

PPL6 Resources and Cultural Infrastructure 

PCS1 Culture [12],[24],[25] 
PCS2 Governance 

PCS3 Awareness 

PCS4 Strategy 

PCS5 Management Commitment 

TCH1 Infrastructure [12],[24],[25] 


TCH2 Security 
TCH3 Networking 





TCH4 Data 
TCHS Telecommunication 
SVQI Responsiveness [26],[16],[14] 


SVQ2 Availability 

SVQ3 Functionality 

SVQ4 Extension 

SVQ5 Reliability 

SVQ6 Efficiency 

SVQ7 Effectiveness 

ITIR1 Technology Management [15], [27] 
ITIR2 IT skills 

ITIR3 IT Partnership 

ITIR4 Quality Improvement 
ITIRS IT acquaintance 
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Figure 2. The critical study of the processional and causal model 


























IT Content 











Input 

Dimension 

Infrastructure 
Center 





Efficiency 











People 























Process 

















Technology 








Service Level 
Agreement 


Process Dimension-1 


IT Implementation Focuss 


Reliability 





Sevice Quality 











Process Dimension-2 


User Satisfaction 


















ITIR 








Output 
Dimension 


























Information Integration Validity Data Integration Compatible Communication 
Access Needs Quality of Service Availability Reachability Infrastructure 
IT Utilization Otomation Execution Time Accurration Transparancy 











Figure 3. Conceptual framework ITIR 
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2.2. Research procedure 

Figure 4 shows two primary stages of research, namely, preliminary study and pilot study. 
This study develops a conceptual framework [28] and a research model [8], and validates the model 
quantitatively. In particular, this pilot study was conducted based on the recommendations from previous 
studies in order to validate model and variable which are developed quantitatively. 





(1) (2) 
Preliminary Studies Pilot Study 


Research 
Design 










Research 
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Research 
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Figure 4. The research procedure 
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2.3. Population, sample and data collection procedure 

The population in this pilot study is stakeholders of IT utilization in higher education institutions in 
Indonesia that cover top managers, middle managers, IT unit managers, and IT staff. The types 
of stakeholders chosen refer to key informant aspects [29, 30]. 250 data were obtained from institutions 
and purposive sampling techniques from universities that have implemented IT, by selecting 160 (64%) 
respondents. The biggest number of respondents (51%) are university graduates and 68% of them have 
experiences less than ten years in managing IT at HEI. Moreover, the highest percentage of their job 
positions belongs to IT staff members (50%). In the data collection procedure, electronic questionnaires are 
sent to 300 email addresses and messages broadcast through social media. 


2.4. Research instruments and data analysis 

The instrument of this study is a survey questionnaire using a five-point Likert scale ranging from 
‘strongly disagree’ to ‘strongly agree’ [31, 32]. In data analysis process, descriptive analysis was conducted 
to produce demographic information to answer the objectives of the first study and clarify the next inferential 
findings. Meanwhile, inferential mode is done using PLS-SEM with SmartPLS 2.0 to assess measurement 
and structural models. Statistical software is used due to strong exploration and prediction with small size 
of sample sizes [33, 34]. The assessment of measurement model covers indicator reliability, internal 
consistency reliability, convergent validity, and evaluation of discriminant validity to test the external model. 
In addition, the assessment of structural model consists of path coefficient (B), determinant coefficient (R7), 
t-tes, effect size (f°), predictive relevance (Q”), and relative impact (q?) checks to evaluate the inside factor. 


3. THE ANALYSIS RESULT 
3.1. The result of the descriptive analysis 

The information of the survey result is presented in Table 2, and the analysis results illustrate that 
61% of respondents stated that IT implementation in HEI aims to meet operational requirements, 17% said to 
fulfill managerial requirements and 22% revealed for strategic requirements. The other result stated that 85% 
of HEI have IT implementation strategic planning. The analysis result of IT implementation architecture 
ownership in HEI reached 59%, while the analysis results of IT implementation roadmap ownership 
reached 52%. 
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Table 2. The IT implementation profile 








Measure Item % 
Goal IT implementation Operational requirements 61% 
Managerial requirements 17% 
Strategic requirements 22% 
Ownership of Strategic Plans for IT Implementation Available 85% 
Not Available 15% 
Ownership of Architecture IT Implementation Available 59% 
Not Available 41% 
Ownership of Roadmap IT Implementation Available 52% 
Not Available 48% 





3.2. The result of the inferential analysis table 

Inferential statistics use a random sample of data taken from a population to describe and make 
conclusions about a population. The results of the inferential analysis are presented in Table 3. Preliminary 
Discriminant Validity is testing performed in two ways, namely by looking at the value of cross loading or 
cross loading Fornell-Lacker's, by comparing the value of the root of AVE (top values in the table) where 
the value must be greater than the correlation between the constructs to construct another. Information about 
Preliminary Discriminant Validity is presented in Table 4. 


Table 3. Results of the measurement model assessments 
Cross Loading 











Tnd L INC ITC ITIR PCS PPL SVQ TCH AVE R 
INCI 0.943 0.943 0.604 0.805 0.807 0.871 0.879 0.823 
ee 0.951 0.951 0.596 0.797 0.788 0.867 0.851 0.861 0.885 0.959 
INC4 0.929 0.929 0.584 0.766 0.801 0.819 0.818 0.826 
ITC] 0.919 0.602 0.919 0.586 0.570 0.605 0.605 0.630 
ITC2 0.948 0.578 0.948 0.609 0.565 0.574 0.577 0.591 
ITC3 0.899 0.546 0.899 0.524 0.486 0.539 0.529 0.528 
ITC4 0.924 0.551 0.924 0.588 0.553 0.564 0.595 0.576 0.867 0.979 
ITCS 0.963 0.606 0.963 0.640 0.555 0.604 0.593 0.633 
ITC6 0.943 0.612 0.943 0.613 0.518 0.588 0.588 0.602 
ITC7 0.922 0.622 0.922 0.632 0.538 0.585 0.595 0.607 
ITIR1 0.933 0.821 0.604 0.933 0.840 0.887 0.882 0.846 
ITIR2 0.885 0.682 0.567 0.885 0.752 0.761 0.798 0.789 
ITIR3 0.947 0.775 0.643 0.947 0.811 0.814 0.808 0.812 0.862 0.969 
ITIR4 0.947 0.811 0.605 0.947 0.821 0.850 0.818 0.825 
ITIRS 0.928 0.801 0.572 0.928 0.797 0.832 0.802 0.814 
PCS1 0.896 0.738 0.496 0.756 0.896 0.805 0.759 0.779 
PCS2* 
PCS3 0.932 0.799 0.525 0.801 0.932 0.794 0.816 0.823 0.830 0.951 
PCS4 0.918 0.764 0.522 0.787 0.918 0.797 0.832 0.810 
PCS5 0.899 0.791 0.574 0.816 0.899 0.809 0.786 0.810 
PPLI 0.909 0.804 0.571 0.825 0.820 0.909 0.850 0.853 
PPL2 0.927 0.848 0.589 0.823 0.799 0.927 0.846 0.851 
PPL3 0.944 0.859 0.550 0.821 0.798 0.944 0.864 0.833 0.854 0.967 
PPL4 0.906 0.800 0.587 0.824 0.816 0.906 0.806 0.809 
PPL5 0.935 0.872 0.584 0.837 0.828 0.935 0.845 0.870 
SVQI 0.925 0.803 0.561 0.813 0.794 0.833 0.925 0.832 
SVQ2 0.923 0.815 0.572 0.802 0.830 0.830 0.923 0.825 0.851 0.981 
SVQ3 0.934 0.863 0.556 0.804 0.783 0.846 0.934 0.862 ` ` 
SVQ4 0.939 0.847 0.589 0.833 0.858 0.853 0.939 0.864 
SVQ5 0.922 0.824 0.561 0.805 0.793 0.820 0.922 0.812 
SVQ6 0.947 0.886 0.620 0.848 0.865 0.884 0.947 0.874 
SVQ7 0.899 0.808 0.559 0.792 0.784 0.841 0.899 0.819 
SVQ8 0.920 0.821 0.625 0.837 0.787 0.849 0.920 0.828 
SVQ9 0.892 0.824 0.559 0.819 0.776 0.810 0.892 0.812 
TCH1 0.924 0.839 0.606 0.844 0.827 0.871 0.814 0.924 
TCH2 0.929 0.830 0.620 0.793 0.822 0.820 0.801 0.929 
TCH3 0.961 0.871 0.590 0.849 0.855 0.892 0.878 0.961 0,867 0.970 
TCH4 0.917 0.788 0.578 0.808 0.819 0.831 0.854 0.917 
TCH5 0.925 0.808 0.590 0.804 0.792 0.833 0.877 0.925 
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Table 4. Preliminary discriminant validity results 
INC ITC ITIR PCS PPL SVQ TCH 

INC 0.911 

ITC 0.641 0.931 

ITIR 0.835 0.644 0.928 

PCS 0.869 0.572 0.854 0.893 

PPL 0.914 0.623 0.894 0.878 0.924 

SVQ 0.909 0.627 0.886 0.877 0.912 0.922 

TCH 0.894 0.641 0.881 0.879 0.913 0.907 0.931 











3.2.1. The result of the measurement model assessments 

The outlined information from this assessment demonstrates statistically that the external model 
shows good psychometric properties with two refusal indicators (INC3 and PCS2). This means that 
the assessment can proceed to the assessment of structural model. 

- Testing Individual Item Reliability, that based on testing values on the outer loading, all values have met 

the threshold value of 0.7, so that there are no indicators deleted, in this case. 

Testing Internal Consistency Reliability, this test is done by looking at the composite reliability (CR) value 

with a threshold above 0.7. Testing results show that all CR values of all variables meet the requirements 

and are valid to be used in the research model. 

- Convergent Validity Test, this test is done by looking at the average variance extracted (AVE) value with a 
threshold value of 0.5. The test results in which all AVE values of all variables have met the requirements 
and are valid for use in the research model. 

- Testing Discriminant Validity, it is tested through cross loading comparison analysis with AVE squared 
value, as follows: there is still found a smaller AVE root value compared to the correlation between other 
variables, in the next step there are removed some indicators. After several tests, there are 2 (two) deleted 
indicators, namely: PCS2 and INC3, with each value of 0.846 and 0.858. By removing these two indicators, 
it is necessary to re-examine internal consistency reliability, composite reliability (CR), convergent 
validity, and discriminant validity. 


3.2.2. The result of the structural model assessments 

In the analysis phase of the structural model, there are six stages of testing, namely testing path 
coefficient (B), coefficient of determination (R°), t-test using the bootstrapping method, effect size (f°), 
predictive relevance (Q?), and relative impact (q°). 
- Testing path coefficient (B), it is done by looking at a threshold value above 0.1 where the path can be 
declared to have an influence on the model if the result of the path coefficient test value is above 0.1. 
The results of the 14 pathways in the research model, 9 of these pathways have significant influence 
and 5 paths have no significant influence as shown in Figure 5 and Table 5. 
Testing coefficient of determination (R°), this test is intended to explain the variance of each endogenous 
variable target with a measurement standard of around 0.670 which is expressed as accurate (A), around 
0.333 is expressed as moderate (M) and 0.190 or below indicates weak variant level (L) as shown 
in Table 5. 
- Testing t-test, it is carried out using the bootstrapping method on SmartPLS 3.0, and the received value on 
the t-test is above 1.96. The test result shows that from a total of 14 hypotheses there are 6 accepted 
hypotheses and 8 rejected hypotheses as shown in the above table as shown in Figure 5 and Table 5. 
Testing effect size (f°), this test is done to predict the influence of certain variables. The threshold values 
are 0.02 for a small effect (k), 0.15 for an intermediate effect (m), and 0.35 for a big effect (b) as shown 
in Table 5. 
Testing predictive relevance (Q?), which is conducted through blindfolding method to explain that certain 
variables used in the model have predictive relationship with other variables in the model with a threshold 
value above zero (0). The testing results reveals that the (Q?) value of all variables above zero (0) indicates 
predictive linkages as shown in Table 5. 
Testing relative impact(q’), which carried out by blindfolding method. This measurement is done to 
measure the relative influence of a predictive link between a variable and other variables. The threshold 
used is equal to f? which is around 0.02 for a small effect, 0.15 for a middle effect, and 0.35 for a big effect 
as shown in Table 5. 
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Table 5. The structural model assessments hypothesis 





Hypothesis 5 A 3 Analysis 
Hip Path B iis R f Q q B t-test R? 
H1  INC->ITIR -0.105 0.845 0.852 0.006 0.653 0.003 Insig R 
H2 INC->PCS 0.801 10.087 0.724 1.396 0.550 0.684 Sig 
H3 INC -> PPL 0.852 16.366 0.825 2.490 0.637 0.978 Sig 
H4 INC->SVQ_ 0.844 15.992 0.820 2.381 0.624 0.911 Sig 
H5 INC->TCH 0.806 15.133 0.800 1.956 0.626 0.857 Sig 
H6 ITC->ITIR 0.098 1.745 0.852 0.033 0.653 0.011 Insig 
H7 ITC->PCS 0.075 0.867 0.724 0.013 0.550 -0.003 Insig 
H8 ITC->PPL 0.084 1.593 0.825 0.022 0.637 0.007 Insig 
H9 ITC->SVQ 0.093 1.372 0.820 0.031 0.624 0.013 Insig 
H10 ITC->TCH 0.131 2.262 0.800 0.052 0.626 0.023 Sig 
H11 PCS->ITIR 0.225 1.708 0.852 0.060 0.653 0.017 Sig 
H12 PPL->ITIR 0.347 2.689 0.852 0.087 0.653 0.029 Sig 
H13 SVQ->ITIR 0.265 1.806 0.852 0.053 0.653 0.017 Sig 
H14 TCH ->ITIR 0.155 1.280 0.852 0.020 0.653 0.005 Sig 
Notes: sig: significant; insig: insignificant; R: rejected; A: accepted; a: accurate; S: small; L: large 











i) 
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Figure 5. Results of the SmartPLS Analysis 


4. DISCUSSION 

First of all, the descriptive analysis reveals that IT need is operationally more than 60%, and even 
the majority of respondents already have IT development strategic plan (85%). As matter of fact, planning 
has come to the technical stage, in which IT implementation architecture reaches 59%, while the ownership 
of IT implementation roadmap reaches 52%. Strategically and operationally, These phenomenon is not 
surprise when referring to HEI’s needs for IT. This is consistently in line with IT planning strategic theory 
which shows that the achievement of HEI’s vision can be realized through IT implementation [35]-[37]. 

Second, in spite of the measurement model assessments represented statistically the good 
psychometric properties, with the indication that there are only two rejected indicators, namely INC3 and 
PCS2. It should be noted that the indicator rejection affects the created model. Therefore, it is necessary to 
develop instruments so that the data collected does not have biased information [38], in addition, it is 
necessary to consider the knowledge and environment of the respondents, because they can affect the results 
of the questionnaire too [39], [40]. So that, it is recommended to focus and minimize this problem in 
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subsequent research by increasing sample size and analyzing systematic errors by attention to research 
design, participants and data collection. 

Third, the similar tendency was found across the results of the four structural model assessments 
(B,t — tes,R*,q*). The five of the 14 paths had statistically insignificant effects in B assessment and 
rejected in t-test examination, small predictive influences (f?) and small relative impact of the predictive 
relevance (q*), especially related to INC and PCS variables. In the case of INC and PCS, there are 
inconsistencies with the procession and causal concepts of the readiness and ZEN framework as the basis for 
model development. In addition, PCS cases are not consistent with e-readiness theory [41] so that it is not 
surprising to note that the majority of respondents (85%) stated that IT has been used operationally more than 
61%. This inconsistency may be related to the development of a model or cultural problem that applies in the 
institution being sampled. 


5. CONCLUSION 

The research finding shows research sustainability from the previous studies until the pilot study. 
Readiness study is conducted to develop a framework and conceptual model, and to validate the model 
quantitatively. In particular, this pilot study elaborates the status of IT implementation readiness in HEI 
and the factors that influence it. In addition, the result of this pilot study can be used to complete the model; 
by paying attention to the research design, participant, data collection and data measurement, data analysis 
and publication of the analysis results. 

Moreover, there are two limitations that researchers must keep in their minds. First, research finding 
cannot be generalized to other institutions because data is influenced by the condition of an institution being 
studied. Second, although the involvement of stakeholders in IT implementation is intended to obtain 
the completeness of the research results in connection with key information aspects, the stakeholder’s 
involvement of each institution may also be somewhat different regarding certain issues presented in 
the survey instrument. Therefore, the subjective conditions of institution are out of control for possible 
analysis results in this study. 

In addition, there are two main important points in this study. First, the status of IT implementation 
readiness in HEI can take to be one of the practical consideration points for sampled institutions policy 
makers regarding the availability of strategic planning and PCS issues to ensure that IT implementation can 
be carried out properly and correctly. Second, the use of INC can be rejected with respect to insignificant 
pathways, hypotheses rejection, and influences that have little impact and predictive relevance in future 
studies. Thus, further research can adopt this these findings, by reconsidering the limitations of this study. 
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